Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize-for-streams-detection #2538

Closed
wants to merge 1 commit into from
Closed

optimize-for-streams-detection #2538

wants to merge 1 commit into from

Conversation

ByiProX
Copy link

@ByiProX ByiProX commented Mar 20, 2021

  1. Optimize repeated images detection when loads streams.
  2. Add option arg skip_step for stream detection instead of reading every 4th frame.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhancement of the webcam stream processing in YOLOv5 by allowing customizable frame skipping.

📊 Key Changes

  • Introduced a new skip_step option to specify how many frames to skip in a video stream before processing.
  • In detect.py, LoadStreams now receives skip_step to control frame skipping.
  • LoadStreams class in utils/datasets.py is modified to skip frames according to skip_step.
  • Added a mechanism to prevent processing the exact same image frame repeatedly in webcam streams.

🎯 Purpose & Impact

  • Allows Efficient Processing: Users can now fine-tune how often frames are processed from a stream, enabling a balance between performance and real-time detection needs. 🚀
  • Prevents Redundant Calculations: By avoiding repeated processing of identical frames, it saves computational resources and time, benefiting environments where image scenes change infrequently. 💡
  • Customizable for Different Use Cases: Applicable in various scenarios, from surveillance to live streaming, where users might prefer different frame rates for object detection. ⚙️

@glenn-jocher
Copy link
Member

glenn-jocher commented Mar 23, 2021

@ByiProX thanks for the PR!

The intended use-case for LoadStreams() is to connect to multiple cameras for batched inference, i.e. you have 8 RTSP cameras, you pass their addresses in a streams.txt file (one per line), and then detect.py will load them all asynchronously and run batched inference on them:

python detect.py --source streams.txt

This results in an incredibly fast multi-camera inference system, but a few issues are:

  • Some cameras may suffer connectivity issues, so one image in the batch will be static, or fail to load, while the other cameras operate correctly.
  • Queue buildup if the system is not realtime capable. The cv2 loader will load the previous frame of the RTSP stream, not the current frame. So for example if your connection is slow and you can only download 1 image per second, the inference stream will never be realtime, it will always be running inference on past frames.

Ideally we'd want a streamloader that checks for and pulls the current frame at a proposed framerate, does not build up a queue, and skips inference if the entire batch is identical as you propose here.

@glenn-jocher
Copy link
Member

@ByiProX see also #2222 for a recent PR to patch a frame loss bug on multi-RTSP streaming.

@glenn-jocher
Copy link
Member

/rebase

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the Stale label Apr 23, 2021
@github-actions github-actions bot closed this Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants